DOCUMENT RESUME 



ED 414 920 



IR 056 778 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Neal, James G. 

The Use of Electronic Scholarly Journals: Models of Analysis 
and Data Drawn from the Project Muse Experience at Johns 
Hopkins University. 

1997 - 04 - 00 

9p . ; Paper presented at the Conference on Scholarly 
Communicating and Technology (Atlanta, GA, April 24-25, 

1997) , see IR 056 774 . 

Association of Research Libraries (ARL) Web site: 
http : //www.arl . org/scomm/scat/ 

Reports - Descriptive (141) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Academic Libraries; Access to Information; ^Electronic 
Publishing; Evaluation Criteria; ^Evaluation Methods; 
Feedback; Higher Education; Humanities; Mathematics; Models; 
*Scholarly Journals; Social Sciences; Use Studies; *User 
Needs (Information) ; User Satisfaction (Information) 
Electronic Resources; * Johns Hopkins University MD 



ABSTRACT 



This paper outlines a series of quantitative and qualitative 
models for understanding and evaluating the use of electronic scholarly 
journals, and summarizes data based on the experience of Project Muse at 
Johns Hopkins University and early feedback received from subscribing 
libraries. Project Muse is a collaborative initiative between the Press and 
the libraries at Johns Hopkins University to provide network- based access to 
scholarly journals including titles in the humanities, social sciences and 
mathematics. Project Muse has been produced from the outset for usability, 
with a focus on user-centered features. This has evolved as a participative 
and interactive process, soliciting input and feedback from users, and 
integrating user guidance components into the system. Six models of use 
analysis are discussed in this paper which cover both the macro or 
library- level and the micro or individual user- level activity: (1) 

subscribing organizations; (2) subscriber behaviors; (3) user demography; (4) 
user behaviors; (5) user satisfaction; and (6) user impact. Project Muse's 
success will ultimately be determined by its support for the electronic 
scholarly publishing objectives outlined in the Association of Research 
Libraries/Association of American Universities (ARL/AAU) work. (AEF) 
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This paper outlines a series of quantitative and qualitative models for understanding and 
evaluating the use of electronic scholarly journals, and summarizes data based on the experience 
of Project Muse at Johns Hopkins University and early feedback received from subscribing 
libraries. 
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Project Muse is a collaborative initiative between the Press and the libraries at Johns 
Hopkins University to provide network-based access to scholarly journals including titles in the 
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humanities, social sciences and mathematics. Launched with electronic versions of forty titles 
still published in print, Project Muse coverage has now been expanded to include electronic-only 
publications. Funded initially by grants from the Mellon Foundation and the National 
Endowment for the Humanities, Project Muse seeks to create a successful model for electronic 
scholarly publishing characterized by affordability and wide availability. It has been designed to 
take advantage of new technical capabilities in the creation and storage of electronic documents. 
It has been developed to provide a range of subscription options for individual libraries and 
consortiums. It is based on a very liberal use and re-use approach that encourages any 
non-commercial activity within the bounds of the subscribing organization. 

Project Muse has been produced from the outset for usability, with a focus on user-centered 
features. This has evolved as a participative and interactive process, soliciting input and 
feedback from users, and integrating user guidance components into the system. An online 
survey is available to all users and libraries are providing information about the local 
implementation and the results of campus and community focus group discussions on Project 
Muse. As the number of subscribing libraries expands and the activity grows, a valuable 
database of user experiences, attitudes and behaviors will accumulate. A new feature will be the 
ability to track and analyze individual search sessions and to observe closely user activities. This 
will enable monitoring the impact of new capabilities and the efficiency of searching practices. 

Six models of use analysis are discussed in this paper which cover both the macro or 
library-level and the micro or individual user-level activity: 

1 . subscribing organizations - which libraries are subscribing to Project Muse and how do 
they compare with the base of print journal customers 

2. subscriber behaviors - how do libraries respond as access to electronic journals is 
introduced and expanded, and in particular, how are acquisitions like Project Muse 
accommodated in service and collection development programs and budgets 

3. user demography - what are the characteristics of the individual user population, in such 
areas as status, background/experience, motivation, attitudes and expectations 

4. user behaviors - how do individuals respond to the availability of scholarly materials in 
electronic format as they explore the capabilities of the system and execute requests for 
information 

5. user satisfaction - what objectives do users bring to network-based access to scholarly 
information, and how do users evaluate system design and performance and the quality of 
search results 

6. user impact - how are user research and information-seeking activities being shaped by 
access to full-text journal databases like Project Muse 

One of the objectives of Project Muse is to achieve full cost recovery status by the 
completion of the grant funding period in 1998. Therefore, it is important to monitor the growth 
in the base of subscribing libraries and to evaluate the impact on the print journal business of the 
Press. An analysis of those libraries subscribing to the full Project Muse database as of June 
1997 (approximately 400 libraries) demonstrates a very significant expansion in the college, 
community college and now public library settings with very low or no history of subscriptions 
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to the print journals. The result is a noteworthy expansion in access to Hopkins Press titles with 
70 percent of the subscribing libraries currently purchasing less than 50 percent of the titles in 
print, and over one-fourth acquiring no print journals from the Hopkins Press. 



PROJECT MUSE 
SUBSCRIBING LIBRARIES 



ARL Universities 
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65 


libraries 


Other Universities 


- 


128 


libraries 


Liberal Arts Colleges 


- 


101 


libraries 


Community Colleges 


- 


53 


libraries 


Public Libraries 


- 


3 


library systems 



PROJECT MUSE 

CUSTOMER PRINT SUBSCRIPTIONS 

No. of Print Percentage of 

Subscriptions Libraries 
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27.8 
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7 . 6 


5-9 


6.8 


10-14 


15.8 


15-19 


13 .0 


20-24 


11.8 


25-29 


9 .0 


30-34 


6.2 


35-40 


2 .0 



One of the explanations for these patterns of subscription activity is the purchase 
arrangement for Project Muse. Over 90 percent of the libraries are subscribing to the full Project 
Muse database of 42 titles. And due to very favorable group purchase rates, nearly 80 percent 
of Project Muse subscribers are part of consortial contracts. The cooperative approach to 
providing access to electronic databases by hbraries in a state or region is widely documented, 
and the Project Muse experience further evidences this phenomenon. 

Another objective of Project Muse is to enable hbraries to understand the use of collections 
and thus to make informed acquisitions and retention decisions. The impact on collection 
development behaviors will be critical, as libraries do indicate intentions to cancel print 
duplicates of Muse titles and to monitor carefully the information provided on individual 
electronic title and article activity. Use information is beginning to flow to subscribing hbraries, 
but there is no evidence yet of journal cancellations for Hopkins Press titles. 

An important area of analysis is user demography, that is the characteristics of the 
individuals searching the Project Muse database. An online user survey and focus group 
discussions are beginning to provide some insights: 

— The status of the user, that is undergraduate student, graduate student, faculty, staff, 
community member, or library employee. As Project Muse is introduced, library staff are 
typically the heaviest users, followed by a growth in student use as campus awareness and 
understanding expands. 
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— Type of institution, that is research university, comprehensive university, liberal arts 
college, community college, or public library setting. As Project Muse subscriptions have 
increased and access has extended into new campus settings, there has initially been 
heavier use in the research universities and liberal arts colleges where there is either 
traditional awareness of Project Muse titles or organized and successful programs to 
promote availability. 

-- The computer experience of users, that is familiarity with searching full-text electronic 
databases through a Web interface. Project Muse users tend to be knowledgeable Internet 
searchers who have significant comfort with Web browsers, graphical presentations of 
information, and constructing searches in textual files. 

-- The location of use, that is in-library, on-campus in faculty office and student residence 
hall, or off-campus. Preliminary data indicates that the searching of Project Muse is taking 
place predominantly on library-based equipment. This can be explained by the inadequate 
network infrastructure that persists at many campuses or the general lack of awareness of 
Project Muse until a user is informed by library staff about its availability during a 
reference exchange. 

-- The browsers used to search the Project Muse database. An analysis of searches over 
an 18-month period confirms that Netscape browsers are used now in over 98 percent of 
the database activity, with a declining percentage of Lynx and other non-graphical 
options. 

Project Muse enables searching by author, tide, or keyword, in the table of contents or 
full-text of the journals, and across all the journals or just selected titles. All articles are indexed 
with Library of Congress subject headings. Hypertext links in table of contents, articles, 
citations, endnotes, author bibliographies, and illustrations allow efficient navigation of the 
database. User searching behavior is an important area for investigation, and some preliminary 
trends can be identified: 

— The predominate search strategy is keyword, with author and tide inquiries occurring 
much less frequendy. This can be partially explained by the heavy undergraduate student 
use of the database and the rich results enabled by keyword strategies. 

— Use of the database is equally distributed across the primary content elements: table of 
contents, article abstracts, images linked to text, and the articles. An issue for future 
analysis is the movement of users among these files. 

-- Given the substantial investment in the creation of LC subject headings and the 
maintenance of a structured thesaurus to enhance access to articles, their value to search 
results and user success is being monitored carefully. 

-- With the expansion of both internal and external hypertext links, the power of the Web 
searching environment is being observed, the user productivity gains are being monitored, 
and the willingness to navigate in an electronic journal database is being tested. 

-- Users are directed to the Project Muse database through several channels. Libraries are 
providing link s from the bibliographic record for titles in the online catalog. Library Web 
sites highlight Project Muse or collections of electronic journals. Subject pages list the 
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Project Muse titles that cluster in a particular discipline. 

— Users are made aware of Project Muse through a variety of promotional and 
educational strategies. Brochures and point-of-use information are being prepared. In 
some cases, campus media have included descriptive articles. Library instructional efforts 
have focused on Project Muse and its structure and searching capabilities. 

— Printing and downloading to disk are important services linked to the effective use of 
Project Muse, given the general unwillingness of users to read articles online. Libraries 
have an interest in maximizing turnover on limited computer equipment, and are focused 
on implementing cost-recovery printing programs. 

— Project Muse is increasingly enabling users to communicate with publishers, journal 
editors, and the authors of articles through e-mail links embedded in the database. 
Correspondence has been at a very low level, but is projected to expand as graduate 
student and faculty use increases and familiarity and comfort with this feature expands. 

With other 400 subscribing libraries and over three million potential users of Project Muse 
in the communities served, it is possible to document global use trends and the changing 
intensity of searching activity: 



PROJECT MUSE 
GLOBAL USE TRENDS 



4th Quarter 1996 



1st Quarter 1997 



1,833,692 

19,922 

199 

9,214 



Requests 
Per Day 
Subscribers 
Per Subscriber 



2,618,069 

29,090 

322 

8,112 



(+42.8%) 

(+46.0%) 

(+61.8%) 

(- 12 . 0 %) 



The progression of use over time as a library introduces access to Project Muse is being 
monitored. Early analysis suggests that the first two quarters of availability produce low levels 
of use, while third quarter use expands significantly. 

User satisfaction with the quality and effectiveness of Project Muse will be the central factor 
in its long-term success. Interactions with users seek to understand expectations, response to 
system design and performance, and satisfaction with results. The degree to which individuals 
and libraries are taking advantage of expansive fair use capabilities should also be gauged. 

Project Muse has focused on various technical considerations to maximize the dependability 
and efficiency of user searching. Detailed information on platforms and browsers is collected, 
for example, and access denials and other server responses which might indicate errors are 
automatically logged and routed for staff investigation. 

Expectations for technology are generally consistent: more content, expanded access, 
greater convenience, new capabilities, cost reduction, and enhanced productivity. It will be 
important to monitor the impact of Project Muse in the subscribing communities and to assess 
whether it is delivering a positive and effective experience for users. 
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Over the now thirty-plus years of library automation activities, we have learned about those 
conditions which improve the positive impact of technology, and Project Muse and its 
implementation must respond to these needs: 

— if the computer package is more decentralized so that is in the hands of users (control); 

— if more developed computing capacity is available (power); 

-- if users have greater competency and experience with computing (training); 

— if the developers are more responsive to the expressed needs of the user regarding 
design and operation of systems (support); and 

— if users routinely rather than selectively use computers and networked-based 
information systems (opportunity). 

A recent study carried out jointly by the Association of Research Libraries and the 
Association of American Universities on electronic scholarly publishing identified a series of 
critical performance attributes for technology: ease of use, timeliness, responsiveness, accuracy, 
authenticity, predictability, adaptability, relevance, eligibility, cost, recovery, innovation and 
extensibility. These qualitative characteristics are essential benchmarks for evaluating Project 
Muse. 

Charles Hildreth, an early investigator of online library catalog systems, established a series 
of analyses which serve as well our review of full-text databases. Hildreth cited five components 
for understanding the user interface: 

— physical, including the input-output equipment, and the structure and location of the 
workstation; 

-- organizational, including the institutional setting, the availability of staff assistance, and 
the provision of user aids; 

-- personal, including the abilities, experience, objectives and needs of the user; 

-- communications, including the language and techniques of interaction; and 
-- functional, including control of operations, search formulation and output. 

Hildreth views user support in terms of five system qualities: easy to use, friendly and 
cordial, protective and forgiving, reliable and responsive, and adaptive and flexible. These 
various elements can be summarized in terms of three general characteristics: audience 
suitability or the degree to which effective use of the system is self-explanatory; metaphorical 
consistency or the extent to which a logical framework is provided; and display legibility. 
Systems which strive to support the user reflect a concern for these elements and emphasize 
both simplicity of design and searching power. 

It is also important to maximize the core advantages of using information in digital formats: 
-- accessibility, that is delivery to locations wherever users can obtain network 
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connections 

— searchibility, that is the range of strategies that can be used to draw relevant 
information out of the database 

-- currency, that is the ability to make publications available much earlier than is possible 
for print versions 

— researchability, that is the posing of questions in the digital environment that could not 
even be conceived with print materials 

-- interdisciplinarity, that is the ability to conduct inquiries across publications in a range 
of diverse disciplines and discover new but related information 

-- multimedia, that is access to text, sound, images, video in an integrated presentation 

— linkability, that is the hypertext connections that can be established among diverse and 
remote information sources 

— interactive, that is the enhancement of user control and influence over the flow of 
information and the communication that can be integrated into the searching activity 

Project Muse will be evaluated against these quantitative and qualitative models. It's success 
will ultimately be determined by its support for the electronic scholarly publishing objectives 
outlined in the ARL/AAU work: 

-- foster a competitive market for scholarly publishing by providing realistic alternatives 
to prevailing commercial publishing options 

— develop policies for intellectual property management emphasizing broad and easy 
distribution and reuse of material 

-- encourage innovative applications of information technology to enrich and expand the 
means for distributing research and scholarship 

-- assure that new channels of scholarly communication sustain quality requirements and 
contribute to promotion and tenure processes 

-- enable the permanent archiving of research publications and scholarly communication in 
digital formats 
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For additional information about the conference, or The Andrew W. Mellon Foundation 's scholarly 
communication initiatives, please contact Richard Ekman . For additional information about ARL or this 
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web site contact Patricia Brennan . ARL Program Officer at (202) 296-2296. 
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